Release 10.1A: OpenEdge Development:
ProDataSets

Buffer BATCH-SIZE and LAST-BATCH attributes

If you want to use the OFF-END event to support transparently retrieving successive batches of data from the server, you need to be able to specify how many rows to fill the table with for each batch, and also signal when all the data from the Data-Source has been retrieved.

The BATCH-SIZE attribute on a ProDataSet temp-table buffer lets you determine how many rows to fill at a time. If you set this integer attribute to a non-zero value, then whenever you execute a FILL on that buffer or its ProDataSet, Progress copies no more than BATCH-SIZE rows from the Data-Source to the temp-table. If the BATCH-SIZE is reached before the end-of-data condition on the Data-Source, the FILL stops for that buffer.

If the end of data is reached before the BATCH-SIZE is exceeded, then Progress sets the buffer’s logical attribute LAST-BATCH to true. If the number of rows remaining to be read exactly matches the value of BATCH-SIZE, Progress detects this and sets LAST-BATCH even without attempting to position beyond the last row of data. In this way, LAST-BATCH is set correctly when the total number of rows that satisfy the buffer’s query is an exact multiple of the BATCH-SIZE.

LAST-BATCH is automatically set during the execution of the FILL method when the BATCH-SIZE on the buffer is non-zero and the end of the data is reached, and is cleared before the start of any FILL or EMPTY actions on the same buffer or ProDataSet. If this default behavior is not satisfactory, it can be set programmatically, right after the FILL or in an AFTER-FILL event. LAST-BATCH is always false when BATCH-SIZE is zero or the Unknown value (?), which means that data batching is not being done.

This attribute is marshaled along with the rest of the ProDataSet definition when the ProDataSet is passed as a parameter to another Progress procedure, whether local or remote. If you define an OFF-END event handler for the buffer in the other procedure, it can check the LAST-BATCH flag for the buffer and determine whether it should make a call to get more rows or if there are no more rows to retrieve. The "Setting up an event handler for the OFF-END query event" section shows an example of this.

Batch-size is primarily intended to limit the number of rows added to a top buffer in a ProDataSet, or to a non-top buffer whose parent table normally has only one row in it. However, it can be set at any level of the hierarchy. The counter used to compare the rows read against the BATCH-SIZE for a buffer is reset for every FILL action. If a buffer is not a top-level buffer in the ProDataSet, then FILL may be called on it many times, once for each parent row. The BATCH-SIZE limit is applied anew each time. For example, if you have ttCust and ttOrder as parent and child, and you put a BATCH-SIZE of 10 on ttOrder, then for each ttCust record that is added to ttCust during the FILL, you can have up to 10 ttOrder records added to the ProDataSet. Thus in a case such as this the BATCH-SIZE on the child table limits the number of child rows for each parent, not the total number of rows for the FILL.

Note: If there are multiple FILLs on the same temp-table, either because FILL is invoked multiple times in succession, or because FILL is invoked on a buffer that is a child of some parent buffer in a ProDataSet and the parent buffer has more than one row, the value of LAST-BATCH will be true if the last invocation of the FILL for that buffer reached the end of data, but false otherwise, and therefore cannot really be relied on. For example, let’s go back to the case of a ProDataSet with ttCust and ttOrder tables, and multiple Customers and their Orders being read in the same FILL. If there is a BATCH-SIZE for ttOrder, then within a single ProDataSet FILL, that BATCH-SIZE may be reached for Orders of some Customers but not for others. LAST-BATCH will be true if the last Customer had more Orders than the BATCH-SIZE for ttOrder, and false otherwise. This is why we say that BATCH-SIZE is really intended for use on a top-level buffer, or on a child buffer when there is only one parent being FILLed at a time.

Though the LAST-BATCH attribute is marshaled with the ProDataSet, the BATCH-SIZE is not marshaled as part of a ProDataSet remote or local parameter. This value must be set and kept on the server side. Normally the client will need to know the value of LAST-BATCH, which is marshaled, but not be concerned about the BATCH-SIZE.